Will LLMs ¢¢, 33, 


Spark a moral 
revolution? 


John Danaher (with Henrik Skaug Seetra) 


Do moral beliefs and practices 
Change over time? 


If You Are Not a Liberal When 
You Are Young, You Have No 
Heart, and If You Are Nota 
Conservative When Old, You 
Have No Brain 


Old Age 


Youth 


Abolition of Female 
slavery Suffrage 
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We Know that moral revolutions have occurred In the past.... 


We presume they might happen again in the future... 


— We are here 


Moral Revolution 


some reasonably significant change In social 
moral beliefs and practices. 
Axiology 


What is good/bad’ | 


Ueontology 
What ts right/wrong’? 


A change in what A change in what 
people believe to be people believe Is 
good and bad. right/wrong., 


Axiological revolution Deontological revolution 


Speed 


Scope 


Can technology hasten/cause a 
moral revolution’ 


The stirrup, by giving lateral support in addition to the front 
and back support offered by pommel and cantle, 
effectively welded horse and rider into a single fighting unit 
capable of violence without precedent. The fighter’s hand 
no longer delivered the blow: it merely guided It. The 
stirrup thus replaced human energy with animal power, 
and immensely increased the warriors ability to damage 
his enemy. Immediately, without preparatory steps, it made 
possible mounted shock combat, a revolutionary new way 
of doing battle. 
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White 1962, p 2 
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The Technological Mediation of Morality 


; 


The central idea is that technologies-in-use help to 
establish relations between human beings and their 
environment. In these relations, technologies are not 
merely silent ‘intermediaries’ but active ‘mediators’ that 
Nelo to constitute the entities that have a relationship 
‘through’ the technology... ...By organizing relations 
between humans and world, technologies play an active, 
though not a final, role in morality. 

Pp, 


Verbeek 2013, 77-78 


Generative Al/LLMs? 


seem to have highly disruptive potential due to the 
number of domains in which they can be 
deployed. They can automate/reduce the cost of 
generating: 


- fexts (including synthetic/evaluative text) 

- Code 

- Video/picture 

- Audio 

Modern society is an information economy in 


which the generation of such outputs Is central to 
much (but not all) of what we do 


Why care about this 
ohenomenon’ 


Try to predict and anticipate possible future moral 
changes and/or offer practical guidance to 
designers or policy makers interested in how 
technology might affect future morality. 


Privacy IS No 
longer a social 
norm. 


Mark Zuckerberg 2010 


Mark Zuckerberg 2010 


People have really gotten 
comfortable not only 
sharing more intormation 
and different kinds, but 
more openly and with 
more people. | nat social 
norm Is just something 
that has evolved over 


time... 
Py, 


Mark Zuckerberg 2010 


Lots of individual case studies and reports about 
technology and moral change. Iry to bring some 
order to the current scholarly landscape. 


There are seven primary mechanisms through which 
technology can change social moral beliefs and 
practices. These mechanisms can be categorised 
into three domains: decisional, relational and 
perceptual. 


Mechanisms of Technomoral Change 


a ae 


Decisional 


How we make 
decisions 


/ \ 


Changes Changes 
option decision 
sets making costs 


Relational 


How we relate to one 
another 


3 4 .) 


New Changes Changes 
relationships —quties/roles duties/roles 


Perceptual 


How we see and 
understand 


/ \ 


New data/ New mental 
information models/ 
metaphors 


Technology Change Options - typically it gives us 
the means of achieving some goal that was 
previously unavailable (i.e. adds options) - 
sometimes it forecloses previously accessible 
options 


K 


Walk to friend’s 
house 


Technology changes the space of options and actions available to 
us. Technology makes options available that once were 


unavailable. Technology can also (less commonly) close off options 
that were once available. 


Cum 


Drive to 
friend’s house 


K 


Walk to friend’s 
house 


Technology changes the space of options and actions available to 
us. Technology makes options available that once were 


unavailable. Technology can also (less commonly) close off options 
that were once available. 


Should | sit silently with my own 
thoughts or distract myself online’? 


Should | respond to that work email 
now, or later’? 


Cum 


Let the LLM 
do it 


ft : 
Do it with 
assistance 


I 


Do it myselt 


The Outsourcing-Assistance Dilemma: Should | do it 


myself or (a) do it with the assistance of an LLM or (b) get 
the LLM to do it all’? 


Technology Changes Decision-Making Costs— 
makes it either harder or easier to (a) access certain 
values and (b) do the right thing. 


Classic example 


Contraception and the ethics of extramarital sex. 


Technology reduced the negative costs of an action 
and thereby enabled people to access a value 
(sexual intimacy) more readily. Not everyone was 
enticed by the reduced cost, and some institutions 
continue to fight against it, but many people have 
now accepted that pre-marital sex is permissible. 


Texts no longer carry the 
same kind of value they 
once had. What’s more it Is 
more difficult to separate 
true from false texts. We 
are like the denizens of 
Borges’ Library of Babel. 
Leads to both practical 
and evaluative linguistic 
nihilism. 
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Low Trust Society 


Lowers the cost of 
‘cheating’ in the 
oroduction of certain 
information. You would be 
a ‘sucker’ not to. Leads to 
an increased expectation 
of this. Could help to foster 
a ‘low trust’ society. 
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Due to lower trust of purel 
textual outputs, there is an 
increased perceived value 
placed on person-to- 
person contact, 
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assessment, verification. : 
Obvious example is 
educational assessment. Ha + 
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Technology Enables New Relationships — transport 
and communications technologies can give us 
access to new human relationship partners. Other 
technologies — Al and robotics being the most 
obvious — can create wholly new non-human 
relationship partners. 


Blake Lemoine Case 


LLMs are understood to 
be actual or potential 
moral others to whom 
we owe some basic (or 
more complex) set of 
moral duties. 


An expanding moral circle’ 


The Present 


Moral circle expansion: A promising strategy to impact the far future 


“,..even If one Is unsure what exactly the future moral 
circle snould look like...ouUshing on the current 
frontiers of the moral circle (e.g. farmed animals) or 
otherwise engendering expansion towards other kinds 
of sentient beings Is a compelling moral priority not 
only for utilitarlans and others concerned with doing 
the most good but, more generally, for those 
concerned with preventing serious wrongs.” 


Reece Anthis and Paez, 2021, p 2 


Or a temporary/permanent 
exclusion’? 


if 


The Present 


Artificial Suffering: An Argument for a Global Moratorium on synthetic Phenomenology 


Until 2050, there should be a global 
moratorium on synthetic 
ohenomenoloagy, strictly banning all 
research that directly aims at or 
Knowingly risks the emergence of 
artificial consciousness on post-biotic 
carrier systems. 


Metzinger 2021, p 1 


Chatbots as Personal 
Avatars/Uploads 


LLMs become 
personalised and allow 
us (or others) to replicate 
ourselves for use In 
multiple contexts and/or 
after we die. The 
schumacher Al interview 
case. 
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Technology Changes the Burdens and Expectations 
Within Relationships — by changing options and 
decison-making costs we have to reevaluate what it 
is that we can, and ought, to do for one another. 
This impacts on our social role-related duties. 


Invention of throwing 
weapons (spears) and 
the evolution of 
cooperative hunting. 


ay 


Joint Mind 
Joint agency/ 
intentionality 
working to a 
common goal 


| should 
treat S2 the 
same as 
myself, if | 
were In 
their 
position 


S1 
Role Responsibility Joint 
| must do X, Y, Z Partners 


towards the common 
goal 


| should 
treat S1 the 
same as 
myself, If | 
were In 
their 
position 


S2 


Role Responsibility 


| must do P, Q, R towards 
the common goal 


The Duty to be Authentic 


At least in some settings, 
some of the time, we have 
a duty to be authentic and 
not use LLMs as 
performance enhancers 
(e.g. Reddit forums recently 
demanding people not use 
GP T-generated posts) 


since it is no so easy to ae: oe 
produce synthetic text/ SS = =>. Ser 
video/audio, you lose the ~ SS PP SS Se 
excuse that this is difficult, 
time-consuming. Leads to 
a Red Queen effect - you 

are expected to do more Wiese 
and more (seems likely in eae 
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Have domestic labour-saving 
technologies changed the 
burdens and expectations of 
domestic work’? 


- Yes: reduced time spent on 
domestic chores and 
increased options for work 
outside the home. 


- Not really: increased the 
demand for clean clothes - 
did not alter the gendered 
division of labour. 


Technology Changes the Balance of Power — 
Relationships are rarely perfectly equal. Oftentimes 
one person or one group has more power than the 
other. Technology can either reduce or increase an 
imbalance of power. 


some research suggests 
that those suffering trom 
scarcity (money, food, time) 
undergo a cognitive tax 
which impedes their ability 
to resolve their scarcity 
problems. Could LLMs help 
remove that cognitive tax’? 
(Imperative to help/reduce 
this) 


SENDHIL MULLAINATHAN 
ELDAR SHAFIR 


carcit 


‘A captivating book that just 
might change the way you live’ 


Cognitive Polarisation 


Those that own the 
infrastructure certainly gain 
power, relative to the rest of 
us, but also it may be that 
the technology is best 
exploited by a cognitive 
elite not suffering from 
other forms of scarcity: I.e. 
the cognitively rich get 
richer. 


Technology Changes Metaphors and Mental Models 
— technology can provide us with new mental 
models and analogies for understanding the world, 
which can change our moral perception of the 
world. 


Social media + smartphone 
Life is not just something to be experienced; it 
Is something to be shared, archived, 
monetised (etc) - changes the perceived 
intrinsic and instrumental value of experiences 
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Tools-to- Theories Heuristic 


We use tools to form 

models of how the world 
works, e.g. mechanical | 
model of the universe 

(clock) or the 

computational model of the 

mind. These can have 

moral implications/ 

interpretations. 


Stochastic Parrot Model 


You are not a stochastic parrot 
-> or maybe that’s all you are’? 
The model is intended to 
deflate our interpretation of 
the technology but this may 
backfire and deflate our 
interpretation of what it means 
to be human. 


Longtermist Shift 


Our moral thinking gets 
displaced into the future 
because of what the 
technology portends (the 
“sparks of AGI”) 


Thank You 


For your attention 


John Danaher 
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